New Approach Of Constant Resolving Of Analytical Programming

This papers’ aim is to provide the Artificial Intelligence community with a better tool for symbolic regression. In this paper, the method of analytical programming and constant resolving is revisited and extended. Nowadays, analytical programming mainly uses two methods for constant resolving. The first method is meta-evolution, in which the second evolutionary algorithm is used for constant resolving. The second method uses non-linear fitting algorithm. This paper reveals the third method, which use the basic mathematics to generate constants. The findings of this study have a number of important implications for future practice. INTRODUCTION A lot of application in symbolic regression field requires some kind of algorithm for constant generation (Koza 1992), (O’Neill, Brabazon & Ryan 2002). Therefore the effective numeric constant resolving algorithm is very important. In this paper, we use analytical programming as a method for symbolic regression (Zelinka, Davendra, Senkerik, Jasek & Oplatkova 2011). This method based on existence of any kind of evolutionary algorithm which generate a pointers to function tables. Analytical programming (AP) from these pointers maps and assemble a final regression function. In the analytical programming algorithm, two main approaches can be selected for constant resolving. The first approach is meta-evolution, in which the second or slave evolutionary algorithm is used for constant resolving. The second approach is to use of nonlinear fitting algorithm. Both approaches added to analytical programming a lot of complexity. This study introduces a new approach for constant resolving in analytical programming algorithm. This approach is founded on basic mathematical calculation. Section II is devoted to the original algorithm of analytical programming. Section III presents the new approach for constant resolving. Section IV presents the methods used for this study. Section V summaries the results of this research. Finally, Section VI presents the conclusions of this study. Differential Evolution Differential Evolution is an optimization algorithm introduced by Storn and Price in 1995, (Storn & Price 1995). This optimization method is an evolutionary algorithm based on population, mutation and recombination. Differential Evolution is easy to implement and has only four parameters which need to be set. The parameters are: Generations, NP, F and Cr. The Generations Parameter determines the number of generations; the NP Parameter is the population size; the F Parameter is the Weighting Factor; and the Cr Parameter is the Crossover Probability, (Storn 1996). In this research, the differential evolution is used as an analytical programming engine. Analytical Programming Analytical Programming, is a symbolic regression method. The core of analytical programming is a set of functions and operands. These mathematical objects are used for the synthesis of a new function. Every function in the analytical programming set core has its own varying number of parameters. The functions are sorted according to these parameters into General Function Sets (GFS). For example, GFS1par contains functions that have only one parameter e.g. sin(), cos(), or other functions. AP must be used with any evolutionary algorithm that consists of a population of individuals for its run (Oplatkova, Senkerik, Zelinka & Pluhacek 2013). The function of analytical programming can be seen in Figure 1. In this case, Evolutionary Algorithm is Differential Evolution. The initial population is generated using Differential Evolution. This population, which must consist of natural numbers, is used for analytical programming purposes. The analytical programming then constructs the function on the basis of this population. This function is evaluated by its Cost Function. If the termination condition is met, then the algorithm ends. If the condition is not met, then Differential Evolution creates a new population through the Mutation and Recombination processes. The whole process continues with the new population. At the end of the analytical programming process, it is assumed that one has a function that is the optimal solution for the given task. Proceedings 30th European Conference on Modelling and Simulation ©ECMS Thorsten Claus, Frank Herrmann, Michael Manitz, Oliver Rose (Editors) ISBN: 978-0-9932440-2-5 / ISBN: 978-0-9932440-3-2 (CD) Fig. 1. Scheme of Analytical Programming with Differential Evolution algorithm ORIGINAL ALGORITHM Let’s have the individual of the n length ind = (x1, x2, . . . , xn) where xi ∈ R . This individual is then rounded indr = (‖x1‖, ‖x2‖, . . . , ‖xn‖) where ‖.‖ is nearest integer function. Let’s have a set called GFSall which consists of m functions GFSall = {{f1, fp1}, {f2, fp2}, . . . , {fm, fpm}} where fm is function and fpm is number of parameters of function fm. Then these functions are sorted to 4 sets : GFS0, GFS1,0, GFS2,1,0 and GFSall. TABLE I. GFSS BY PARAMETERS GFS0 = {{f1, 0}, {f2, 0}, . . . , {fn, 0}} ⊂ GFSall GFS1 = {{f1, 1}, {f2, 1}, . . . , {fn, 1}} ⊂ GFSall GFS2 = {{f1, 2}, {f2, 2}, . . . , {fn, 2}} ⊂ GFSall GFS1,0 = GFS1 ∪ GFS0 GFS2,1,0 = GFS2 ∪ GFS1 ∪ GFS0 The sets from table I are expanded to the maximum value from the individual because we expected that the value of each number in individual could be higher then the length of GFS. TABLE II. EXAMPLE OF GFSS, WHEN MAXIMUM VALUE OF INDIVIDUAL IS 6 GFS0 = {{K, 0}, {x, 0}, {K, 0}, {x, 0}, {K, 0}, {x, 0}} GFS1,0 = {{Sin, 1}, {Cos, 1}{K, 0}, {x, 0} {Sin, 1}, {Cos, 1}} GFS2,1,0 = {{Plus, 2}, {Minus, 2}, {Sin, 1}, {Cos, 1}, {K, 0}, {x, 0}} GFSall = {{Plus, 2}, {Minus, 2}, {Sin, 1}, {Cos, 1}, {K, 0}, {x, 0}} Then we need to construct a matrix with functions mapped by individual. After that we have function = ((f1, fp1), (f2, fp2), . . . , (fn, fpn)) where fn is function and fpn is number of parameters of function fn. Then we need to choose which function is applied to which function. The values are pointers to the functions in GFSs. After that, we have constructed function; however, there is a constant K, which have to be resolved. Now we have two possibilities • Meta-evolution e.g. Differential Evolution • Non-linear least square fitting for example Levenberg-Marquardt NEW APPROACH Let’s have the individual of the n length ind = (x1, x2, . . . , xn) where xi ∈ R . This individual is then rounded indf = (⌊x1⌋, ⌊x2⌋, . . . , ⌊xn⌋) where ⌊.⌋ is round to floor. Now we can construct a difference between ind and indf . indc = ind− indf In vector indf are pointers to GFSs and indc are corresponding constants. The decimal numbers in indc are in range < 0, 1 >. These numbers could be easily converted to constants into the chosen range. Let’s have a set called GFSall which consists of m functions GFSall = {{f1, fp1}, {f2, fp2}, . . . , {fm, fpm}} where fi is function and fpi is number of parameters of function fi. Then this functions are sorted to 4 sets : GFS0, GFS1,0, GFS2,1,0 and GFSall. The next key change in analytical programming algorithm is function selection. In this new approach, the GFSs are not expanded to the maximum value of the individual. The selection of function is controlled by modulo function. On the position where the constant K will be mapped; we can read a constant number from the indc vector. The original mapping algorithm is the same. Now we have constructed function with resolved constants. PROBLEM STATEMENT The overall research question to be answered within the study is whether there is a possibility to outperformed the original analytical programming method. This section presents the design of the research question. We performed experiments to get an insight in the constant resolving of analytical programming. The research question of our study can be outlined as follows: RQ: Analysing the impact of new approach on the calculation duration and minimization performance of analytical programming. The research question (RQ) aims to get an insight on the new approach of constant resolving of analytical programming and understand the actual effectiveness of this technique. For this reason, we use 3 different methods for constant resolving. Analytical programming with differential evolution and two new versions of proposed algorithm. Then, we try to outperformed the original constant resolving algorithm of analytical programming. To asses the performance of fitness function, we used descriptive statistics. METHOD New constant resolving algorithm for analytical programming was tested for searching regression functions. Results were compared by descriptive statistics. Following functions have been tested • f(x) = 45.5 • f(x) = 3x+ 0.65 • f(x) = 2.3x − 20x− 5.6 • f(x) = 3.65 ∗ sin(2x) These functions were selected with emphasis on constant resolving. Functions such as constant, linear, quadratic and harmonic were tested. There was generated 20 points for each function. And the task for analytical programming was to fit these points. Three methods of constant resolving were tested. • Analytical programming with differential evolution further referred to as AP+DE • New analytical programming version with constant range < −1000, 1000 > further referred to as AP2(-1000,1000) • New analytical programming version with constant range < 0, 10 > further referred to as AP2(0,10) TABLE III. SYSTEM CONFIGURATION Parameter Value CPU AMD Phenom II X2 3GHz RAM 8 GB Operation system Windows 7 Professional 64 bit Programming language LUA 5.2 Table III shows the system configuration for performing tests. Table IV shows the analytical programming set-up. The number of leafs (functions built by analytical programming TABLE IV. SET-UP OF ANALYTICAL PROGRAMMING Parameter Value Number of leafs 16 GFS functions plus, minus, multiply, divide, power, log, log10, exp, sqrt, floor, ceil, abs, sin, cos GFS constants x, K can be seen as trees) was set to 16. This value was sufficient for the purpose of this paper. TABLE V. SET-UP OF DIFFERENTIAL EVOLUTION Parameter Value NP 45 Generations 300 F 0.2 Cr 0.9 Table V shows the set-up of differential evolution. The best set-up of differential evolution is the subject of further research. A. Fitness function Fitness function used for this task is as following: